reader/cloudtrail: improve sync process #698
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
JIRA: https://jira.qiniu.io/browse/PDR-7022
1. 检查判断非法的 interval 和 concurrent 选项值 2. 优化日志打印 3. 修复 syncSingleFile 的 doneChan 消费者线程无法退出,每次 sync 就会泄露一个 4. sync 之后准确地返回哪些文件是真正完成 sync 了的,然后再写进入到 metastore 里 +5. 每当同步到一定数量的文件之后就写入到 metastore,降低丢失的概率